An information theoretic method for designing multiresolution principal component transforms

Principal Component Analysis (PCA) is concerned basically with finding an optimal way to represent a random vector through a linear combination of a few uncorrelated random variables. In signal processing, multiresolution transforms are used to decompose a time signal into components of different resolutions. In this paper, we consider designing optimal multiresolution transforms such that components in each resolution provide the best approximation to the original signal in that resolution. We call a transformation that admits this optimality property a Principal Component Multiresolution Transform (PCMT). We show that PCMTs can be designed by minimizing the information transfer through their basic building blocks. We then propose a method to do the minimization in a stageby-stage manner. This latter method has a great appeal in terms of its computational simplicity as well as theoretical interpretations. In particular, it agrees with Linsker’s principle of self organization. Finally, we provide analytic arguments and computer simulations to demonstrate the efficiency of our method.